-
Notifications
You must be signed in to change notification settings - Fork 661
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(reaction_analyzer): add reaction anaylzer tool to measure end-to-end delay in sudden obstacle braking response #5954
feat(reaction_analyzer): add reaction anaylzer tool to measure end-to-end delay in sudden obstacle braking response #5954
Conversation
eaff394
to
ecf4c0f
Compare
356b216
to
4cb81a3
Compare
79b9056
to
ae041f9
Compare
5d693bc
to
5cc6583
Compare
efdd814
to
7d7adba
Compare
7103a8c
to
995bab5
Compare
@mitsudome-r -san, I guess, I found the problem. When we start to publish the empty pointcloud before the initialization, it causes the crash above. We should publish the pointcloud message after the vehicle is initialized. I tried 7 times in planning_control mode and I didn't get the error we mentioned above. When you have time, could you test it again with the latest commit? |
Very nice work @brkay54
Also, I did not encounter any node crashes like in the last comment. I think the README file was quite descriptive. Overall, it's a well-done piece of work! |
…-end delay in sudden obstacle braking response Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
…_pointcloud_with_object Signed-off-by: Berkay Karaman <[email protected]>
…ability Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
…ine latency, update readme Signed-off-by: Berkay Karaman <[email protected]>
Signed-off-by: Berkay Karaman <[email protected]>
18a452d
to
3b6ef51
Compare
@beyzanurkaya Thank you for the review.
I have added a condition for checking whether the output path is valid or not at the beginning of the node.
Thank you for the suggestion. I added a debug marker for the entity that will be spawned.
Because the vehicle is not moving in perception_planning mode, the control topics may not react. Thus the reaction_analyzer waits for them to react. I added this warning in the readme, I also added a warning message prints reaction_analyzer is waiting for which node. I appreciate your review. Could you also try with the last commit? |
3b6ef51
to
98df256
Compare
… reaction_chain Signed-off-by: Berkay Karaman <[email protected]>
c10fb5c
to
f74c83f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…-end delay in sudden obstacle braking response (autowarefoundation#5954) * feat(reaction_analyzer): add reaction anaylzer tool to measure end-to-end delay in sudden obstacle braking response Signed-off-by: Berkay Karaman <[email protected]> * feat: implement message_filters package, clean up Signed-off-by: Berkay Karaman <[email protected]> * feat: update style and readme Signed-off-by: Berkay Karaman <[email protected]> * feat: add predicted path for the PredictedObject and add publish_only_pointcloud_with_object Signed-off-by: Berkay Karaman <[email protected]> * feat: add wrong initialize localization protection, improve code readability Signed-off-by: Berkay Karaman <[email protected]> * feat: launch occupancy_grid_map from reaction analyzer's own launch file Signed-off-by: Berkay Karaman <[email protected]> * feat: update Signed-off-by: Berkay Karaman <[email protected]> * feat: change function names Signed-off-by: Berkay Karaman <[email protected]> * feat: update Signed-off-by: Berkay Karaman <[email protected]> * feat: improve style, change csv output stringstream Signed-off-by: Berkay Karaman <[email protected]> * fix: ci/cd Signed-off-by: Berkay Karaman <[email protected]> * feat: update for new sensor setup, fix bug, optimize code, show pipeline latency, update readme Signed-off-by: Berkay Karaman <[email protected]> * fix: container die problem Signed-off-by: Berkay Karaman <[email protected]> * feat: update stats, check path param, add marker, warn user for wrong reaction_chain Signed-off-by: Berkay Karaman <[email protected]> --------- Signed-off-by: Berkay Karaman <[email protected]>
running reaction analyzer but the vehicle is not initializing.. I was able to initialize the vehicle by giving a current pose and goal pose through rviz but the dynamic obstacle is still not spawning in the map. (using the default nishishinjuku map). Is this an ongoing issue or is there a work around for this ? |
Description
Main issue:
Strictly depends on:
Optionally depends on:
The main purpose of the reaction analyzer package is to measure the reaction times of the nodes by listening to the
pre-determined topics. It can be used for measuring the reaction time of the perception pipeline, planning pipeline, and
control pipeline. To be able to measure both control outputs and perception outputs, it is necessary to divide the node
into two running_mode:
planning_control
andperception_planning
.Planning Control Mode
In this mode, the reaction analyzer creates a dummy publisher for the PredictedObjects and PointCloud topics. In the
beginning of the test, it publishes the initial position of the ego vehicle and the goal position to set the test
environment. Then, it spawns a sudden obstacle in front of the ego vehicle. After the obstacle is spawned, it starts to
measure the pre-determined topics to catch the reacted messages of the planning and control nodes. When all the topics are reacted, it calculates the reaction time of the nodes and statistics, and it creates a csv file to store the
results.
Perception Planning Mode
In this mode, the reaction analyzer reads the rosbag files which are recorded from AWSIM, and it creates a topic
publisher for each topic to replay the rosbag. It reads two rosbag files:
path_bag_without_object
and
path_bag_with_object
. Firstly, it replays thepath_bag_without_object
to set the initial position of the egovehicle and the goal position. After
spawn_time_after_init
seconds, it replays thepath_bag_with_object
to spawn asudden obstacle in front of the ego vehicle. After the obstacle is spawned, it starts to measure the pre-determined
topics to catch the reacted messages of the perception and planning nodes. When all the topics are reacted, it
calculates
the reaction time of the nodes and statistics, and it creates a CSV file to store the results.
Related links
Related Issue:
Tests performed
You can see the results in the Excel table here. You can see the test dates below the page.
Also, some tests were shared in issues comments: #5540
Notes for reviewers
This PR currently depends on other PR below:
Interface changes
Effects on system behavior
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.
In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.
Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.
After all checkboxes are checked, anyone who has write access can merge the PR.